The present invention relates to a method for controlling a vehicle system, an analysis module, and a vehicle.
For the purpose of controlling vehicle systems by the user of a vehicle, conventionally, switches provided the user with immediate haptic feedback on an operating action that has been performed. Over the years, the number of vehicle systems to be operated (e.g., light, music systems, navigation systems, telephone systems, velocity control systems, distance control systems, lane departure warning systems, etc.) has increased whereby controlling each of the vehicle systems using its own respective switch has been found to be impractical. In the meantime, rotary/push switches have been predominantly used to control the vehicle systems in combination with a predefined menu structure. Also, it is known to control the vehicle systems using voice inputs. The limited number of input possibilities with a rotary/push switch may require very complex menu structures. On the other hand, ambient noises, for example, of a convertible vehicle traveling through a tunnel may make it difficult to reliably identify voice inputs.
In at least that regard, the present invention is directed to a method for controlling a vehicle system, an analysis module, and a vehicle that avoids at least the above-described disadvantages of the known methods.
In accordance with one or more embodiments of the disclosure, a method for controlling a vehicle system by a user is provided. A sequence of sensor data from a sensor system is received and a sequence of positions is generated by the position of a body part being determined from each element in the sequence of sensor data, where a first position, a second position temporally following the first position, and a third position temporally following the second position are selected from the sequence of positions. Moreover, a first circumferential angle, defined by the first position, the second position, and the third position, is determined. A first circular direction in the clockwise direction is identified if the first circumferential angle is between a first lower threshold value v and a first upper threshold value π−w. A first circular direction in the anticlockwise direction is identified if the first circumferential angle between a second lower threshold value π+v and a second upper threshold value 2π−w. The vehicle system is controlled based on the first circular direction identified, and thus, the vehicle system is operated in an intuitive manner.
The sensor data, for example, may be provided by a sensor system in the form of a camera. The camera may be, for example, a monochrome camera which is sensitive in the range of visible light. A monochrome camera may enable a higher resolution of the sensor data, in particular, a higher resolution of the image data. Moreover, a monochrome camera may exhibit little noise. A color camera may be used when determining the position of a body part. For example, skin tones can stand out well from the background and can make it possible to more easily determine the position of a body part. In another example, an infrared camera may be used. An infrared camera may be favorable in poor light conditions. An infrared camera may also allow the determination of the position of the body part based on the radiated heat. The camera may be a 2-D camera which can be used to obtain a two-dimensional image of the environment and therefore a two-dimensional determination of the position of the body part. Moreover, a 3-D camera may be used as the sensor system so as to determine, for example, the position of the body part in three dimensions, which makes it possible to identify circumferential angles that are in a plane parallel to the camera axis.
The sensor data may be received regularly, for example, in periodical intervals. In one aspect, the sensor data may be queried and/or received at least every 20 ms so as to identify the circle gesture in a robust manner. The vehicle system may be controlled in a precise manner if, for example, the sensor data is queried at least every 12 ms. If the interval of time between the reception of the sensor data is limited to more than 5 ms, the noise in the sensor data may be reduced by a low-pass filter.
According to one or more embodiments of the disclosure, the movements of different body parts may be used to control the vehicle system. For example, the position of a finger, such as the thumb or the index finger may be used. The position of a finger can be determined using sensor systems. In one example, the use of the position of the wrist to control the vehicle system can simplify the movement analysis as a result of the position changes that are typically smoother than a finger. Moreover, in another example, the position of an object held by the user can also be determined instead of the position of a body part.
By way of another example, the first position, the second position, and the third position need not be temporally immediately successive positions in the sequence of positions. For example, the position at the time “t_n” may be used as the first position, the position at the time “t_(n+k)” may be used as the second position and the position at the time “t_(n+k+1)” may be used as the third position.
If a circular direction is not identified with the necessary reliability (e.g., in other words, if neither a circular direction in the clockwise direction nor a circular direction in the anticlockwise direction has been identified), possible incorrect operation of the vehicle system can be avoided.
The preselected values “v” and “w” may be between π/40 and π/20, for example. In one aspect, the values v and w may be identical so that the calculation can be simplified based on the symmetrical boundary conditions.
In one embodiment of the invention, the second position is selected based on the interval of time from the first position and the third position is selected based on the interval of time from the second position. In another embodiment, when the positions are periodically determined, the first position, the second position, and the third position may be selected in advance so the invention can be implemented with little outlay. In yet another embodiment, the intervals of time between the first position and the second position and the intervals of time between the second position and the third position may be selected to be the same.
In accordance with another embodiment of the invention, the second position may be selected based on the spatial distance from the first position, and the third position may be selected based on the spatial distance from the second position.
A spatial distance between the first position and the second position and between the second position and the third position above a certain limit value may increase reliability when identifying a circular movement. A spatial distance of more than 1 cm, for example more than 3 cm, has been found to be reliable. In order to also be able to detect smaller circular movements by a user, the spatial distance between the first position and the second position and the spatial distance between the second position and the third position may be kept shorter than 10 cm.
According to yet another embodiment of the invention, a first circle including a first circle center point, a first circle radius, and/or a first circle plane is determined based on the first position, the second position, and the third position.
The first position, the second position, and the third position may be positions on the circumference of the first circle and the position and size of the circle in two-dimensional or three-dimensional space may be determined. For example, the parameters of the circle may be used to individually control the vehicle system.
According to an embodiment of the disclosure, the vehicle system may be controlled based on the first circle radius. For instance, the volume of a music system is controlled using a first circle, such as a first circle gesture, the first circle radius of which is greater than 10 cm. Another first circle, the first circle radius of which is less than 8 cm, may allow the user to browse through the individual titles in a music album.
Not all movements performed by the user of a vehicle may be intended to control a vehicle system. For example, the driver of the vehicle may use his or her right hand to describe a quadrant gesture in order to reach for sunglasses in the glove compartment or in the center console. As such, the vehicle system may be controlled only when the first circle radius is between 1 cm and 40 cm. A desired circle gesture may typically be assumed with a first circle radius of more than 5 cm. If the circle radius is less than 20 cm, the space to be sensed by the sensor system can be kept to a reasonable size.
According to an additional embodiment of the disclosure, the vehicle system is controlled if the first circle center point is within a predefined activation square or a predefined activation cube. As described above, not every circle-like movement performed by the user of a vehicle is intended to control a vehicle system. For instance, specifying an activation square or an activation square with an edge length of 5 cm to 15 cm, e.g., 10 cm, may reduce the risk of unintentional operation of the vehicle system. A plurality of activation squares or activation cuboids may be provided that are spatially separate from one another and may be used to control different vehicle systems or different functions of the same vehicle system.
In accordance with another embodiment, the vehicle system is controlled based on the orientation of the first circle plane with respect to a predefined plane. The orientation of the circle plane may provide a further degree of freedom for operating the vehicle system. When operating a music system, for example, a first circle plane, the surface normal of which is oriented parallel to the vehicle longitudinal axis, may allow the user to browse through the individual tracks in a music album. Alternatively, a first circle plane, the surface normal of which runs parallel to the vehicle transverse axis, may allow the user to select a particular music album.
In accordance with yet another embodiment, the vehicle system may be controlled based on a first circle sector angle which is defined by the first position, the first circle center point, and the third position. Specifying a first circle sector angle above a predetermined threshold may increase the certainty that a desired gesture of the user is present. The circle sector angle may also be used as a parameter for controlling the vehicle system. For example, in the instance of a small circle sector angle, the map from a navigation system may be enlarged to a lesser extent than in the instance of a larger circle sector angle.
According to a further embodiment, the vehicle system is controlled based on a first average circle sector angular velocity between the first position and the third position. As such, a further parameter for controlling the vehicle system may be obtained. In one example, a high first average circle sector angular velocity, for example above 2π per second (2π/s), may allow a user to browse through music titles with spacings of 10 music titles. Moreover, a low circle sector velocity, for example below 0.5 π/s, may allow a user to browse through the individual music titles.
According to another embodiment, the vehicle system may be controlled based on the length of a first circumferential section between the first position and the third position. For example, the length of the first circumferential section between the first position and the third position may be perceived by a user as a natural measure for changing a parameter of a vehicle system. The aforementioned embodiment can therefore make it possible to operate the vehicle system in an intuitive manner.
According to a yet another embodiment, the vehicle system is controlled based on a first average circumferential velocity between the first position and the third position. Users may perceive the first average circumferential velocity, which is also influenced by the first circle radius in addition to the first circle sector velocity, as a more natural specification of the velocity at which a parameter of a vehicle system is intended to be changed. This configuration can therefore make it possible for users to quickly adapt.
In a further embodiment of the disclosure, the vehicle system provides a fourth position temporally following the first position, a fifth position temporally following the fourth position, and a sixth position temporally following the fifth position to be selected from the sequence of positions, where a second circumferential angle, defined by the fourth position, the fifth position, and the sixth position, is determined. A second circular direction in the clockwise direction is identified if the second circumferential angle is between a lower threshold value “a” and an upper threshold value “π−b.” A second circular direction in the anticlockwise direction is identified if the second circumferential angle is between a lower threshold value “π+a” and an upper threshold value “2π−b.” For example, the vehicle system is controlled based on the match between the first circular direction and the second circular direction. The three additional positions may improve the robustness when identifying the circle gesture and may therefore reduce the risk of incorrect operation. Even though only one additional set of three positions (e.g., fourth position, fifth position, sixth position) is described herein, it may be understood that more sets of three positions may be used, which may be used continuously in order to continuously check the plausibility of the control inputs received.
In yet another embodiment, a second circle includes a second circle center point, a second circle radius, and/or a second circle plane is determined based on the fourth position, the fifth position, and the sixth position. The vehicle system may be controlled based on the distance between the second circle center point and the first circle center point and/or the difference between the second circle radius and the first circle radius, and/or the deviation of the second circle plane from the first circle plane.
The vehicle system may be controlled, for example, only when the distance between the second circle center point and the first circle center point is less than a predetermined circle center point distance, such as 10 cm.
According to another embodiment, a pose may be determined based on the sensor data for the first position and/or for the second position and/or for the third position, and for the driver assistance system to be controlled based on the identified pose. A pose may be, for example, the extension of a particular number of fingers or of particular fingers of a user. For example, the volume may be changed only when it is identified that the user is extending an index finger. Poses may not be randomly struck by a user and may therefore be a good indicator of whether or not the user desires to control the vehicle system.
An analysis module including at least one processor for executing stored instructions to carry out a method for controlling a vehicle system, as described in detail above, may allow a user to operate a vehicle system in an intuitive manner.
A vehicle which has a sensor system and an analysis module described above may increase traffic safety by reducing risks of distractions that are caused by complicated operations in a vehicle system.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.